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13*  Abstract  -  continued 

the  amount  of  time  spent  on  each  presentation  was  controlled;  3)  In  both 
methods,  repetition  produced  gains  In  number  of  key  Items  correctly  re¬ 
ported  and  In  overall  quality  of  the  gist.  In  summary,  the  results  Imply 
that  on-line  glating  Is  a  feasible  and  economical  conslc eratlon  for  timely 
message  contents  reporting;  the  best  method  of  glating  v  tries  with  type  of 
material;  and  that  there  la  a  need  to  identify  the  properties  of  a  message 
which  determines  Its  dlfflcul  v  in  order  to  choose  the  moat  effective  glat¬ 
ing  method  for  use  in  a  given  comnunlcatlon. 
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IIS.  AStTMACT 1 

The  prlmscy  objective  of  research  conducted  by  the  MONITOR  SKILLS  Work  Unit 
of  the  Behavior  and  Systems  Research  Laboratory  is  to  improve  performance  In  Army  moni¬ 
tor  Jobs  with  special  emphasis  on  developing  and  testing  new  work  methods.  An  importanf 
segment  of  ther/research  it  concerned  with  human  factors  studies  In  communication  analy¬ 
sis  processing.  The  present  technical  research  note  report^  on  two  experimental  method^ 
of  glstlng  --  reporting  in  brief  form  the  essential  Information  in  a  communication  -- 
In  comparison  with  baseline  performance  In  simulated  on-line  glstlng.  Twenty-four  com¬ 
munications  processors  were  each  tested  using  two  methods  of  glstlng.  In  one  method 
(free  repeat),  the  processor  listened  once  to  the  complete  message  with  no  option  to 
stop  or  replay,  preparing  as  complete  a  gist  as  possible,  and  was  then  free  to  replay 
the  tape  aa  he  judged  necessary  to  complete  the  gist.  In  the  second  method  (forced 
repeat),  the  processor  listened  to  the  complete  message  five  additional  times  with  no 
option  to  stop  or  replay.  The  resulting  gists  were  evaluated  in  terms  of  key  Items  of 
information  correctly  reported,  quality  of  gist  as  rated  by  four  judges,  and  in  the 
case  of  the  free  repeat  method  the  number  of  times  the  tape  was  stopped  and  replayed  in 
the  glstlng  process. 

The  two  work  methods  of  glstlng  examined  in  this  study  were  designed  so  that  the 
effect  of  repetition  on  the  accuracy  and  quality  of  the  gist  could  be  determined.  Re¬ 
sults  of  the  experiment  showed  1)  the  free  repeat  method  produced  a  higher  quality  gist 
and  a  higher  percentage  of  key  items  correctly  reported;  2)  less  time  was  used  In  pro¬ 
ducing  a  complete  gist  in  the  free  repeat  method;  however,  in  the  forced  <?peat  method, 
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FOREWORD 


The  complex  weapons  and  surveillance  systems  of  the  modem  Army  have  created 
a  relatively  new  series  of  jobs  requiring  operators  to  monitor  instrument  panels, 
radarscopes,  communication  nets,  and  other  types  of  signal  detection  apparatus. 
The  MONITOR  SKILLS  Work  Unit  of  the  Behavior  and  Systems  Research  Labora¬ 
tory  deals  with  the  many  personal,  environmental,  and  situational  variables  affecting 
human  performance  in  the  detection  and  analysis  of  a  broad  variety  of  signals.  The 
primary  objective  is  to  improve  performance  in  Army  monitor  jobs,  with  special  em¬ 
phasis  on  developing  and  testing  new  work  methods. 

An  important  segment  of  the  research  is  devoted  to  human  factors  studies  in  com¬ 
munication  analysis  and  processing.  Technical  Research  Note  236  reports  on  two  ex¬ 
perimental  methods  of  gisting-that  is.  abstracting  the  essential  elements  of  a  voice 
message-in  comparison  with  baseline  performance  in  simulated  on-line  gisting. 

The  entire  work  unit  is  responsive  to  objectives  of  RDT&E  Project  2Q02470 1A723, 
"Human  Performance  Experimentation,”  FY  1972  Work  Program. 
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A  COMPARISON  OF  TWO  METHODS  OF  GISTING 
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BRIEF 


Requirement: 

To  evaluate  two  methods  of  gi  sting-reporting  in  brief  form  the  essential  information 
in  a  communication-in  terms  of  accuracy  in  reporting  key  items  and  quality  of  the  sum* 
mary  produced. 


Procedure: 

Twenty-four  communications  processors  were  each  tested  using  two  methods  of  gist- 
ing.  In  both  methods,  the  subject  listened  to  the  complete  message  one  time  with  no 
option  to  stop  or  replay,  preparing  as  complete  a  gist  as  he  could.  In  one  method,  he 
was  then  free  to  replay  the  message,  starting  and  stopping  as  he  judged  necessary  to 
complete  the  gist  (free  repeat).  In  the  other  method  (forced  repeat),  the  subject  lit¬ 
tered  to  the  complete  brassage  five  additional  times  with  no  option  to  stop  or  replay. 
The  resulting  gists  were  evaluated  in  terms  of  key  items  of  information  correctly  re^ 
ported,  quality  of  gist  as  rated  by  four  judges,  and  in  the  case  of  the  free.repeat 
method  the  number  of  times  the  tape  was  stopped  and  replayed  in  producing  a  gist 


Findings: 

The  free  repeat  method  i»roduced  a  higher  quality  gist  and  a  higher  percentage  of 
key  items  correctly  reported  than  did  the  forced  repeat  method. 

The  free  repeat  method  also  took  less  time  to  produce  a  complete  gist.  However,  , 
using  the  free  repeat  method  the  gister  could  proceed  at  his  own  pace  after  the  first 
play-through.  Using  the  forced  repeat  method,  the  amount  of  time  spent  on  each  pre¬ 
sentation  was  controlled. 

In  both  methods,  repetition  produced  gains  in  number  of  key  items  reported  cor¬ 
rectly  and  in  overall  quality  of  the  gist.  Amount  of  gain  varied  considerably  with  the 
message. 


Utilization  of  Findings: 

] 

Complete  repetitions  of  a  message  with  no  opportunity  to  replay  segments  does  not 
appear  to  be  an  economical  method  of  gisting.  For  some  information  requirements,  and 
with  some  messages,  on-line  gisting  or  a  gist  prepared  after  one  or  two  repetitions  may 
represent  an  acceptable  tradeoff  between  timeliness  and  quality  of  information. 

Since  messages  vary  in  difficulty,  it  becomes  important  to  identify  the  properties 
which  make  a  message  easy  or  difficult  in  order  to  choose  the  most  effective  gisting 
method  for  use  with  a  given  communication.  , 
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A  COMPARISON  OF  TWO  METHODS  OF  GISTING 


A  major  problem  In  the  field  of  radio-telephone  communications  is 
the  rapid  extraction  of  complete  and  accurate  information  from  the  in¬ 
creasingly  large  volumes  of  Incoming  messages.  Standard  processing 
techniques  are  inadequate  to  cope  with  this  great  mass  of  material  in  a 
rapid  manner.  Currently,  communications  monitors  record  incoming  messages 
on  magnetic  tape  and  transcribers  prepare  written  verbatim  transcripts 
using  these  tapes.  It  has  been  suggested  that  a  new  method  be  developed 
to  replace  this  time  consuming  process.  Gistlng  messages  may  be  a  more 
efficient  method  of  extracting  communications  Information. 

Gisting  is  the  process  of  reporting  the  content  of  a  communication 
in  abstracted  form.  On-line  or  real-time  gisting  would  require  the  com¬ 
munications  monitor  to  extract  the  important  information  from  the  message 
as  it  enters  the  system.  This  procedure  would  result  in  a  saving  in  per¬ 
sonnel  requirements  as  well  as  in  processing  time.  In  addition,  the  gist 
would  provide  in  abstracted  form  all  the  information  needed  to  be  communi¬ 
cated  to  the  next  level  in  the  communications  chain.  Off-line  gisting 
from  taped  messages  would  also  result  in  a  saving  of  processins  time. 

As  a  standard  procedure,  gisting  places  additional  requirements  on 
the  processor.  Gisting  demands  that  the  processor,  in  addition  to  per¬ 
ceiving  and  recognizing  the  words  in  the  message  as  is  required  in  tran¬ 
scription,  filter,  organize,  and  summarize  the  information  these  words 
convey.  The  best  method  for  gisting  from  tapes  may  be  quite  different 
from  the  best  method  for  transcribing.  A  transcriber  frequently  replays 
small  sections  of  a  tape  in  an  effort  to  transcribe  every  word.  In  gist¬ 
ing,  it  is  Important  to  listen  for  thoughts  rather  than  to  put  down  every 
utterance;  it  may  therefore  be  more  efficient  to  listen  to  the  entire 
message  or  long  segments  of  it. 

Two  work  methods  for  gisting  were  examined  in  the  present  study. 

The  main  purposes  were  to  establish  1)  which  of  two  work  methods  is  the 
better  way  to  gist;  2)  the  effect  of  repetition  on  the  accuracy  of  the 
gist,  and  3)  a  base-line  estimate  of  the  quality  and  quantity  of  infor¬ 
mation  extracted  from  the  first  presentation  of  a  message. 


METHOD 


Subjects 

Twenty-four  communications  processors  with  field  experience  in  tran¬ 
scription  of  foreign  language  communications  served  as  subjects. 


I 


Test  Tape 


Two  English  language  conmunications,  referred  to  as  "messages  1  and 
2,"  and  each  approximately  5  minutes  in  length,  were  recorded  on  separate 
tapes  under  studio  conditions.  The  conversations  were  based  on  typical 
foreign  language  messages.  The  speakers,  although  untrained  in  diction 
or  announcing,  made  an  effort  to  speak  clearly  and  distinctly.  The  taped 
conversations  were  rated  by  experienced  conmunications  processors  as  far 
superior  to  the  average  communication. 


Apparatus 

The  tapes  were  reproduced  on  Midwestern  AN/TNH-11  recorder/ repro¬ 
ducers^  and  presented  through  Telex  HTW-2  electrical  headsets.  The  re¬ 
corder/reproducers  were  modified  so  that  reversals  of  the  tape  could  be 
counted.  Counts  were  recorded  on  Veeder-Root  120  DC  volt  electro-mechani¬ 
cal  counters.  However,  some  of  the  modified  pieces  of  equipment  intro¬ 
duced  a  distracting  noise  and  had  to  be  eliminated  from  the  study.  There¬ 
fore,  replay  data  were  not  obtained  for  one-third  of  the  subjects. 


Test  Procedure 

Two  gisting  work  methods  were  examined  in  this  study — method  A,  free 
repeat  and  method  B,  forced  repeat.  In  method  A,  the  processor  listened 
to  the  complete  message  with  no  option  to  stop,  start,  or  replay  segments 
of  the  message  on  the  first  trial.  He  was  requested  to  record  as  complete 
a  gist  as  possible.  Following  the  first  trial,  the  processor  had  the 
option  to  start,  stop,  and  replay  the  tape  as  frequently  as  he  wished, 
until  hr  had  finished  what  he  felt  was  a  complete  gist  of  the  message. 

In  method  B  the  processor  listened  to  the  complete  message  six  times 
with  no  option  to  stop,  start,  or  replay  segments  of  the  communication. 
With  each  repetition,  the  processor  was  asked  to  record  as  complete  a 
gist  as  possible.  Work  produced  on  each  trial  was  available  to  the  pro¬ 
cessor  to  add  to,  correct,  or  alter  during  each  additional  trial. 

A  repeated  measures  2x2x2  design  (method  x  message  x  test 
session)  set  up  as  a  replicated  Graeco-Latin  square  design  was  used  to 
assess  the  effects  of  gisting  methods  A  and  B.  Each  man  was  tested 
using  both  methods  of  gisting.  The  subjects  were  divided  into  four 


i-'  Trade  names  are  used  only  in  the  Interest  of  precision  in  reporting. 
Their  use  does  not  constitute  Indorsement  by  BESRL  or  the  Army. 
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groups  of  six  men  each.  The  complete  Graeco-Latin  square  design  was: 

Testing  Session 
First  Second 


Sublects 

Group  I 

A1 

B2 

Group  II 

B1 

A2 

Group  III 

A2 

B1 

Group  IV 

B2 

A1 

A  and  B  stand  for  methods,  1  and  2  for  messages.  For  example,  group  I 
glsted  message  1  using  method  A  during  the  first  test  session  and  then 
glsted  message  2  using  method  B  In  the  second  testing  session.  In  this 
design,  each  man  serves  as  his  own  control  In  comparisons  of  methods  and 
of  messages. 


Dependent  Variables 

Three  measures  of  performance  were  used: 

1.  Percent  Items  Correct.  The  percent  of  key  items  correctly  re¬ 
ported  in  each  gist. 

2.  Quality  Rating.  Quality  was  defined  in  terms  of  presenting  a 
complete,  accurate,  and  concise  summary  of  all  important  intelligence 
information  contained  In  the  message.  Each  gist  was  rated  on  a  scale 
from  1  to  5  ranging  from  unacceptable  to  superior. 

3.  Tape  Reversals.  The  number  of  times  the  tape  was  stopped  and 
replayed. 


Scoring  Method 

The  gists  were  evaluated  by  a  panel  of  four  judges  selected  on  the 
basis  of  high  performance  and  many  year*.  of  experience  In  the  field  of 
communications  processing.  The  panel  first  Identified  the  key  Items  of 
Importance  In  each  conversation.  Using  this  list  of  items,  each  judge 
Independently  evaluated  the  work  of  each  man  tested.  Each  judge  checked 
the  key  item  aa  right,  wrong,  or  omitted,  and  rated  the  complete  gist  on 
quality.  For  each  man  tested,  the  scores  from  the  four  judges  were  aver¬ 
aged  to  form  a  single  mean  score  for  each  gist  for  each  of  the  first  two 
performance  measures.  The  reliability  of  the  mean  score  across  judges, 
ignoring  differences  In  level  between  judges,  was  determined  by  an  analy¬ 
sis  of  variance  procedure^  for  each  performance  score.  These  reliability 


^Winer,  B.  J.  Statistical  principles  and  experimental  design.  New 
York:  McGr-v-Hill.  1962,  p.  1J1. 
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coefficients  for  percent  Items  correctly  reported  and  for  quality  rating 
are  shown  In  Table  1. 


Table  1 

RELIABILITY  COEFFICIENTS  OF  THE  MEAN  OF  THE  FOUR  JUDGES 
FOR  PERCENT  ITEMS  CORRECTLY  IDENTIFIED  AND  QUALITY 
RATINGS  OF  EACH  TRIAL  FOR  METHODS  A  AND  B. 


_ Group -Reliability _ 

Quality  Rating  $Items  Correctly 

Reported 


Method  A 


Method  B 


Trial  1 

•  51 

•97 

Trial  2 

.82 

•91 

Trial  1 

.60 

•97 

Trial  2 

.60 

.96 

Trial  3 

•  77 

•96 

Trial  4 

.80 

•96 

Trial  5 

•79 

•95 

Trial  6 

•75 

•92 

RESULTS 


Work  Method 

Comparison  of  performance  for  work  method  A,  free  repeat,  and  work 
method  B,  forced  repeat,  revealed  that  method  A  was  generally  superior  to 
method  B  (Figure  1)*'’.  A  method  x  session  x  message  x  groups  of 
subjects  analysis  of  variance  comparing  the  first  trial  of  the  two  methods, 
and  comparing  the  final  repetition  of  method  A  with  each  repetition  of 
method  B,  was  computed  separately  for  each  performance  measured.  Analyses 
of  variance  for  percent  Items  correctly  reported  and  quality  ratings  are 
presented  In  Tables  2  and  3» 


^  See  footnote  2,  page  3» 

^  The  frequency  distribution  for  performance  scores  is  shown  in  the 
Appendix. 


ANALYSES  OF  VARIANCE  COMPARING  THE  FIRST  TRIAL  OF  METHOD  A  WITH  THE  FIRST  TRIAL 
OF  METHOD  B  AND  THE  FINAL  TRIAL  OF  METHOD  A  WITH  EACH  TRIAL  OF 
METHOD  B  FOR  QUALITY  RATING 


Comparison  of  performance  on  the  first  trial  of  each  work  method  re¬ 
vealed  no  significant  differences,  indicating  that  methods  A  and  B  had 
comparable  baselines.  The  percentage  of  items  correctly  reported  on  the 
final  trial  of  method  A  and  the  fourth  and  fifth  repetitions  of  method  B 
did  not  differ  significantly.  However,  the  final  trial  of  method  A  re¬ 
ceived  a  significantly  higher  quality  rating  than  all  trials  of  method  B 

(P  <  -05). 

Groups  of  subjects  were  found  to  differ  significantly  in  all  compari¬ 
sons  except  the  quality  rating  f  jt  the  final  trial  of  method  A  and  the 
fifth  repetition  of  method  B  (p  <  .05).  In  the  experimental  design 

employed,  these  group  differences  are  confounded  with  two-way  interac¬ 
tions.  Most  of  the  variation  between  groups  corresponds  to  the  interac¬ 
tion  between  methods  and  messages.  Study  of  such  interactions  was  not  a 
major  goal  of  the  present  experiment,  and  only  limited  attention  is  given 
to  the  present  result. 

The  method  x  message  interaction  is  shown  in  Figure  2.  With 
method  A,  performance  was  superior  on  message  1,  while  with  method  B  per¬ 
formance  was  superior  on  message  2.  This  result  suggests  that  the  best 
work  method  may  be  determined  by  characteristics  of  the  message.  It  is 
not  clear  what  these  determining  characteristics  are.  As  may  be  seen  in 
Figure  2,  there  was  very  little  difference  in  performance  with  methods  A 
and  B  on  message  2. 

The  percentage  of  items  correctly  reported  in  message  1  was  signifi¬ 
cantly  smaller  than  in  message  2  when  the  first  and  final  trials  of 
method  A  were  compared  to  the  first  trial  of  method  B  (p  <  .05) . 

Message  differences  were  not  significant  in  any  other  comparisons. 

The  significant  interactions  observed  in  Tables  2  and  5*  with  the 
exception  of  method  x  message  Interaction,  were  not  consistent  across 
all  analyses  and  are  not  Important  to  this  discussion. 

Method  A  produced  a  superior  product  in  a  shorter  time  than  did 
method  B.  A  complete  gist  was  prepared  in  26  minutes  average  time  using 
method  A.  With  method  B,  where  the  time  was  controlled,  a  gist  was 
turned  in  approximately  every  8  minutes  following  each  repetition.  Thus, 
it  required  approximately  48  minutes  to  produce  the  final  gist.  Compari¬ 
sons  of  methods  A  and  B  for  percent  items  correctly  reported  indicated 
that  there  was  no  significant  difference  between  the  fourth  trial  of 
method  B  and  the  final  trial  of  method  A.  The  difference  in  time  to 
arrive  at  this  point  was  four  minutes,  in  favor  of  method  A. 


Repetition 

As  expected,  performance  scores  Improved  as  a  direct  function  of 
repetition.  A  trial  x  group  of  subjects  analysis  of  variance  was  com¬ 
puted  for  methods  A  and  B  for  each  performance  measure.  A  summary  of  the 
analyses  of  variance  is  provided  in  Table  4.  Performance  with  method  A 
Increased  significantly  from  a  mean  of  42.5  percent  words  correctly 
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Figure  1.  Comparitoo  of  average  gisting  scores  for  the  first  and  final  trial  of  method  A  with  the 
six  trials  of  method  B  for  percent  items  correctly  identified  and  quality  rating. 


identified  to  a  mean  of  72.0  percent  in  the  final  gist,  and  from  a  mean 
quality  rating  of  i.6  on  the  first  gist  to  3*1  on  the  final  gist. 


With  method  B,  the  mean  percentage  of  words  correctly  identified  in¬ 
creased  from  42.6  percent  on  the  first  gist  to  67*8  percent  on  the  sixth 
gist,  and  the  mean  quality  rating  increased  from  1.3  to  2.8.  Since  the 
analysis  of  variance  revealed  a  significant  main  effect  for  repetition 
(Table  2),  individual  mean  differences  between  repetitions  of  method  B 
were  assessed  by  the  Newman  Keuls  technique  for  each  performance  measure^. 
The  results  of  the  analyses  are  shown  in  Table  5*  There  were  no  signifi¬ 
cant  differences  in  the  percentage  of  items  correctly  Identified  between 
trials  2  and  3*  3  and  4,  4  and  3,  and  3  and  6 .  The  mean  increase  in 
quality  rating  was  not  significant  between  repetitions  3  and  4,  4  and  5, 
and  3  and  6.  All  other  differences  were  significant. 

Using  scores  on  the  first  gist  with  each  method  as  a  base-line 
measure  of  performance,  percentage  gain  in  percent  words  correctly  iden¬ 
tified  and  percentage  gain  in  quality  ratings  as  a  function  of  repetition 
was  computed  using  the  previous  repetition  as  a  base-line.  The  percentage 
gains  are  shown  in  Figures  3  and  4.  In  addition,  percentage  gain  with 
method  A  across  trials  using  the  first  trial  as  a  base-line  have  been 
computed  separately  for  messages  1  and  2  and  are  shown  in  Figure  3* 

% 

The  number  of  tape  reversals  were  counted  for  13  of  the  24  glsters. 
The  frequency  of  reversals  varied  from  3  to  258.  Fourteen  of  the  15 
glsters  made  between  3  and  33  reversals.  These  scores  were  used  to  com¬ 
pute  correlation  coefficients  between  number  "f  reversals  and  1)  percent 
items  correctly  identified,  2)  quality  rating  and  3)  time  to  complete 
gist.  The  correlation  coefficients  were  .3i>  *39  and  .44,  respectively. 

It  does  not  appear  that  frequency  of  reversal  is  a  good  predictor  of 
performance. 


DISCUSSION 

The  two  work  methods  of  glsting  examined  in  this  study  were  designed 
so  that  the  effect  of  repetition  on  the  accuracy  and  quality  of  the  gist 
could  be  determined.  As  expected,  accuracy  Increased  as  a  direct  func¬ 
tion  of  repetition  (Figure  l).  However,  it  appears  that  the  complete 
repetition  of  a  five-minute  message  with  no  opportunity  to  stop  or  repeat 
smaller  segments  (method  B)  is  not  an  economical  method  of  glsting.  With 
method  A,  where  the  gister  was  allowed  to  repeat  segments  of  his  choice, 
a  final  gist  of  higher  quality  was  prepared  in  a  shorter  time. 

With  method  A,  an  average  of  J2  percent  of  the  items  were  identified 
correctly  in  the  final  completed  gist.  The  final  product  was  produced  in 
approximately  28  minutes.  The  gist  completed  after  one  complete  presen¬ 
tation  of  the  message  --  the  mode  which  simulates  on-line  glsting--  was 
nrepared  in  approximately  8  minutes,  and  an  average  of  42  percent  of  the 


^ See  footnote  2,  page  3» 
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items  were  identified  correctly.  That  is,  it  required  three  and  a  half 
times  longer  to  produce  an  absolute  gain  of  20  percent  --  or,  using  the 
£irsti  trial  as  the  baseline,  a  relative  gain  of  69  rercent  items  cor¬ 
rectly  identified.  On  the  first  presentation  of  the  message,  the  average 
quality  rating  was  1.5  (midway  between  unacceptable  and  poor),  and  on  the 
final  version,  the  quality  rating  increased  to  2.1  (fair),  or  a  percentage 
gain  of  99  percent.  The  acceptable  trade-off  between  timeliness  and 
accuracy  probably  varies  with  the  specific  situation.  Therefore,  in  some 
situations,  the  on-line  gist  riay  satisfy  the  requirement  for  timeliness. 

i 

With  method  B‘,  there  were  two  constraints:  1)  The  complete  message 
was  repeated  with  no  opportunity  to  stop  and  repeat  small  segments;  and 
2)  only  three  minutes  were  allowed  after  each  trial  to  complete  the  gist. 
The  time  constraint  between  trials  may  have  been  a  limiting  factor.  It 
may  be  that  gists  of  highier  quality  would  have  been  prepared  if  more  time 
had  been  given  between  trials.  As  shown  in  Figure  3>  performance  contin¬ 
ued  to  improve  through  the  fifth  repetition.  However,  percent  gain  in 
word 8  correctly  Identified  leveled  off  after  the  second  vepetitlon 
(Figure  4). 

With  method  B,  it  is  possible  to  estimate  the  gain  in  information 
per  unit  time.  Each  trin^  required  approximately  8  minutes  to  complete. 
Therefore,  the  tinke  represented  by  the  second  trial  is  twice  that  of  the 
first,  or  2  to  1,  and  the  third  is  3  to  1,  and  so  on  for  each  trial, 

1  using  the  first  trial  as  the  baseline.  As  shown  in  Figure  4,  there  was 
a  22  p'ercent  gain' in  the  percentage  of  Items  correctly  identified  and  a 
30  percent  gain  in  quality  rating  on  the  second  trial  or  in  twice  the 
time  as  the  first  trial.  Two  complete  presentations  may  be  a  feasible 
procedure  for  producing  a  quick  gist. 

The  percent  gain  in  performance  varied  with  messages  (Figure  5)» 
Obviously,  some  messages  profit  more  from  repeated  listening  than  others. 
On  the  basis  of  data  collected  in  the  present  study,  it  is  difficult  to 
account  for  these  differences  between  messages.  Part  of  the  difference 
may  be  explained  by  dlffererces  in  groups  of  subjects;  however,  it  Is 
doubtful  that  this  differn  e  accounts  for  the  entire  effect.  The  dif¬ 
ference  in  messages  makes  ic  difficult  to  generalize  about  the  effect  of 
repetition.  In  some  cases,  repeated  listening  produced  large  gains, 
while  in  others,  the  percent  gain  in  performance  was  relatively  small. 

The  difference  between  methods  A  and  B  varied  with  the  specific 
message  (Figure  2).  For  one  message,  the  highest  scores  were  obtained 
with  method  A,  and  for  the  other  message,  the  highest  scores  were  ob¬ 
tained  with  method  B.  It  was  not  the  purpose  of  the  present  study  to 
specify  the  properties  of  mecsages  which  determine  differences  In  per¬ 
formance;  therefore  no  1  explanation  of  this  interaction  is  provided  in 
these  analyses.  Messages  appear  to  vary  along  many  dimensions,  and 
subtle  differences  between  Messages  in  all  probability  account  for  these 
observed  Interactions.  The  data  collected  cannot  explain  why  one  method 
is  better  for  one  message  an<^  another  method  is  better  for  another.  The 
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messages  used  in  the  present  study  were  chosen  to  represent  typical 
messages  processed  by  communications  operators.  The  two  messages  were 
equal  in  intelligibility,  equally  long,  and  relatively  £ree  from  back¬ 
ground  noise.  However,  they  varied  along  a  number  of  other  dimensions. 
The  topics  of  conversation  were  different,  the  number  of  speakers  varied, 
and  the  rate  of  speaking  may  have  been  unequal.  Other  more  subtle  dif¬ 
ferences  more  difficult  to  measure  may  also  have  existed,  such  as  number 
of  inferences  contained  in  one  message  or  directness  of  expression.  The 
results  of  the  present  study  underline  the  importance  of  identifying  the 
properties  of  a  communication  which  determine  the  difficulty  of  gisting 
and  make  one  message  different  from  another.  The  importance  of  these 
difficulties  in  identifying  the  properties  of  messages  which  determine 
performance  has  been  substantiated  by  other  studies  (Quantitative  evalua¬ 
tion  of  current  procedures  in  voice  processing,  Technical  Research  Report 
1174^ and  a  study  in  progress  on  comparative  gisting  ability  of  LeFox 
Grey  and  standard  systems  personnel). 

With  regard  to  the  general  level  of  performance,  the  average  quality 
rating  obtained  on  the  final  gist  with  method  A  was  j.l,  a  rating  of  fair. 
This  was  the  ilghest  average  obtained.  An  average  of  J2  percent  of  the 
items  were  identified  correctly  on  the  final  completed  gist  with  method 
A--the  highest  average.  These  scores  were  obtained  on  clearly  spoken 
English  language  messages,  and  all  speakers  were  native  English  language 
speakers.  It  is  clear  from  these  results  that  gisting  Involves  more  than 
language  speaking  ability. 

In  summary,  the  results  imply  that  1)  the  best  method  of  gisting 
varies  with  the  type  of  material;  2)  there  is  need  to  identify  the  pro¬ 
perties  of  a  message  which  determine  its  difficulty;  and  3)  on-line 
gisting  is  a  feasible  consideration  for  timely  reporting  of  the  contents 
of  a  message. 


House,  Joyce  L.  and  Stanley  L.  Cohen.  Quantitative  evaluation  of 
current  procedures  in  voice  processing  (U).  Technical  Research  Report 
1174  (AD  5I6989).  Behavior  and  Systems  Research  Laboratory.  July  I97I. 
CONFIDENTIAL 
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Figure  3.  Average  percentage  gain  in  gisting  score  by  trial  by  methods  A  and  B  for  percent  items 
correctly  identified  and  quality  rating. 
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